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ABSTRACT 

The specific focus of this document is on the 
evolutionary development of the Neighborhood Educational Center 
Project supported under Title III, 1965 Elementary Secondary Act, 
which was marked by continual evaluation and appropriate 
modification. Application of the concepts of individualized 
instruction was one of the basic means by which the Neighborhood 
Educational Center (NEC) Project was to achieve its goals. 
Developmental activities required for the implementation of an 
instructional system so oriented included -the production of -three 
essential types of components* behaviorally stated objectives, 
prescriptive packages, and assessment instruments. Student 
performance data on the NEC objectives are summarized and 
disseminated in two ways; individual profiles and cluster profiles. 
These are the bases for information relevant to the implementation, 
evaluation, and modification of the NEC process. Major emphasis 
regarding organization for instruction has been placed upon the 
implementation of the cluster concept as the basic instructional unit 
in the NEC project. The cluster is intended to provide flexibility, 
which in turn supports individualization of instruction. A 
cooperative and supportive relationship exists among the NEC teachers 
and administrative staff. An important outcome of the NEC project is 
the verbalized willingness of teachers and administrators to be 
accountable for student success on the NEC instructional objectives, 
something quite different from accountability for a standardized test 
score. (JM) 
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PART ONE - GENERAL 



I. INTRODUCTION 

Consistent with previous practice, this report is complied In two parts* 
The first part consists of a general description and evaluation of the pro- 
ject with minimal emphasis upon the presentation of data and associated 
technical issues. The second pa.'t consists of a Technical Supplement and 
is intended for those who have interest in the more detailed aspects of the 
evaluation. 

For the reader who Is unfamiliar with the Neighborhood Educational 
Center (NEC) Project and Its previous documentation, an Historical Overview 
is presented in Appendix A. Besides providing a summary orientation to the 
project and its target population, this description portrays the evolution- 
ary nature of the process by which the American Institutes for Research (AIR) 
evaluation staff and the NEC staff worked together to develop and implement 
a successful program. 

During the third project year, the AIR evaluation staff reviewed surveys 
of other programs for the educationally disadvantaged conducted throughout 
the country. This review was designed to provide perspective in viewing 
the characteristics and accomplishments of the NEC Project. Review of these 
surveys revealed that the NEC Project incorporates most, if not all, of the 
features which characterize successful programs and has avoided or overcome 
the major difficulties encountered by unsuccessful programs. In addition, 
the detailed documentation of activities and accomplishments which charac** 
terizes the NEC Project was found to be disturbingly lacking in 90% of the 
projects surveyed. The report of this review (Project Memorandum #3) is 
reproduced as Appendix B. 

II. GENERAL DESCRIPTION AND EVALUATION 
A. The NEC Process 

The evolutionary development of the NEC process was marked by continual 
evaluation and appropriate modification. This development will be discussed 
In terms of five areas of concern; (1) the components of the instructional 
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system, (2) the data for managing the Instructional system, (3) the organl 
zation for Instruction, (k) the relationships among teachers and supervi- 
sors, and (5) the accountability for student progress. 

The Components of the Instructional System 

Application of the concepts of Individualized instruction was one of 
the basic means by which the NEC Project was to achieve Its goals. Devel- 
opmental activities required for the Implementation of an Instructional sys- 
tem so oriented included the production of three essential types of compo- 
nents. By the end of the third project year this effort had resulted In the 
fol lowing: 

1. BEHAV I ORALLY STATED OBJECTIVES - The development of over 600 
Instructional objectives In language arts and mathematics which 
were behavioral ly stated. 

2. PRESCRIPTIVE PACKAGES - The development of at least one prescrip- 
tive package for each of the instructional objectives* 

3. ASSESSMENT INSTRUMENTS - The development of criterion referenced 
measurement instruments for each objective for the purpose of 
diagnosis and the assessment of mastery of the Instructional 
objectives,. 

The behavlorally stated Instructional objectives were compiled with 
the cooperation of the entire teaching staff (approximately 130 teachers),, 
The teachers were each asked to submit a minimum of 10 behavlorally stated 
Instructional objectives. The teachers were not restricted to the curricu- 
lum areas to which they were currently assigned as far as the content of 
these objectives was concerned. The general guidelines for the content of 
the objectives were that they reflect those skills the teachers felt were 
most relevant to the students In the NEC project. The teachers were further 
advised that in no way was it anticipated or necessary that the objectives 
correspond to an external criterion, such as the Stanford Achievement Test 
(SAT), or any particular set of instructional materials. 

The objectives submitted by the teachers were turned over to committees 
of teachers and administrators for editing and classification. The evalua- 
tion staff and the DPS technical monitor screened the objectives In order 



to assure that they were behavioral 1y stated. The committees categorized 
the objectives into 28 curriculum areas. In addition to categorizing the 
objectives, the committees attempted to sequence the objectives such that 
instruction would proceed based upon mastery of prerequisite objectives. 

Prescriptive packages were developed for each instructional objective 
by a committee of teachers and administrators. The prescriptive package 
includes a listing or discussion of the instructional objective and at 
least one recommended way to teach to the objective. Within the prescrip- 
tive packages, most of the objectives are "keyed" to commonly available 
instructional materials and texts. Most of the prescriptive packages also 
contain a listing of the immediate prerequisites for that objective. The 
prescriptive packages are intended to be an optional resource for the 
teacher, i.e., their use is not mandatory. 

Two kinds of assessment instruments were developed for the diagnosis 
of instructional needs and the verification of mastery of instructional ob- 
jectives.* One was an Objective Referenced Test (ORT) , a test which measures 
performance on only one objective. It typically contains no more than 10 
items and takes less than 15 minutes to complete. The other was a Curricu- 
lum Embedded Test (CET), a test which measures performance on approximately 
6 to 10 related objectives and is intended for broad range diagnosis or 
mastery. 

As with the derivation of the Instructional objectives, teachers were 
directly involved in the construction of the ORTs and CETs. For each of the 
objectives which the teachers submitted they were asked to submit test items 
which they felt specifically measured performance on that objective. The 
test items were screened by a committee of teachers and administrators and 
by the evaluation staff. The primary concern In this portion of the screen- 
ing process was the level of vocabulary used and whether or not the item 
was In apparent correspondence with the objective. There was an immediate 
spin-off or benefit from the teachers' involvement in writing the objectives 
and the test items. For instance, prior to writing objectives and items, 
many teachers and administrators wanted to teach initia. language arts 
y 1 ■ ■ ■ — - 

Student instructional needs In the NEC project are defined with respect to 
the set of instructional objectives. In this sense diagnosis and mastery 
are very similar, e.g., In general, If the student does not evidence mastery 
of the "next" objective (a score of 90%), then his "needs" have been diagnosed. 



skills as encoding and decoding. It was discovered that this was too con- 
densed, i.e., too many prerequisite skills were implied. Consequently, 
encoding and decoding were broken down into "oral language", "written 
language", "auditory and visual perception", "phonetic analysis", and 
"structural analysis". The point here is not that one taxonomy Is better 
than the other, but rather that the teachers were involved in a deeper level 
of analysis of what they were trying to teach. In the opinion of the auth- 
ors, this kind of teacher behavior which has continued and expanded (for 
example, into the area of testing) represents one of the most important 
developments In the NEC project. 

The Data for Managing the Instructional System 

Student performance data on the NEC objectives are summarized and dis- 
seminated in two ways, (1) individual profiles, and (2) cluster profiles. 
The profiles are the basis for information relevant to the implementation, 
evaluation, and modification of the NEC process. The purpose here is to 
discuss these two profiles and introduce a rationale for the adoption of a 
third type of prof i le, local criterion profiles. The local criterion pro- 
files will address two issues: (1) increased support to the teacher in 

terms of the planning and management of Instruction, and (2) the establish- 
ment of clearer guidelines by which CAPs can evaluate student progress and 
subsequently evaluate and modify the instructional system. 

The Individual profiles (see Figure 1) indicate the objectives which 
have been mastered by a particular student. The profile matrix Is two di- 
mensional, the rows represent the 28 areas and the columns represent objec- 
tives within each area. The cell entries In the profile, l.e., the numbers, 
represent the number of instructional days that It took for that student to 
master that objective (an asterisk indicates that an objective exists at 
that point which has not been mastered). Following compilation, the Indi- 
vidual profiles are returned to the teacher (cluster). 

The cluster profiles (see Figure 2) summarize the performance of a 
particular cluster on the NEC objectives. There are three cell entries for 
each objective within each area, (1) the mean number of days taken to mas- 
ter the objective by those students who have mastered It, (2) the standard 
deviation about that mean, and (3) the proportion of the students In that 
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cluster who have mastered that objective. While cluster profiles are gener- 
ally available, they are primarily used by the CAP. CAPs attempt to use 
the existing cluster profiles In several ways. First, they try to determine 
whether or not the students In a particular cluster are spread-out on the 
objectives (extreme homogeneity would tend to indicate that little or no 
individualization is taking place). Second, they attempt to analyze the 
depth of student performance within the areas of the NEC objectives. For 
example, they examine whether or not the students In cluster "x" are moving 
as far in addition and subtraction as the CAP feels they should. Third, 
the CAP attempts to ascertain whether or not particular objectives are caus- 
ing trouble, e.g., they are not being mastered In a reasonable period of 
time or by enough students. She may find that particular teachers are having 
difficulty with a particular objective; or, site may find that all of the 
clusters are having difficulty with that particular objective. In the first 
case, she may suspect that the teacher does not understand the objective, 
has not selected a particularly effective teaching approach, etc. In the 
second case, the CAP would suspect that something Is wrong with the system. 
For example, prerequisite objectives have been omitted by the teachers (or 
perhaps they don't exist and will have to be added to the NEC objectives), 
the objective as stated may be ambiguous, the mastery test for that objec- 
tive or prerequisites may be inadequate, etc. In practice, specifics with- 
in the two preceding examples have occurred and modifications where Indicated 
have been made. 

The profiles as described above were consistent with the developmental 
progress of the project. However, as Implementation of the objectives and 
mastery tests became more generalized, the need for more sophisticated or 
definitive feedback to the teachers became obvious. Teachers have reported 
(via teacher questionnaires) that the planning and the actual delivery of 
Instruction to a student body with Increasingly different needs was becoming 
Increasingly difficult (In all probability the variation among students al- 
ways existed but was not so clearly recognized by the teachers until the 
Implementation of Individual profiles). Given that there Is substantial 
variability at each grade level (and subsequently each cluster) among the 
NEC students on the Stanford data, one should have an Intuitive feel for the 
problem stated by teachers; the NEC objectives cover a wider range of per- 
formance and In much greater detail than does a general achievement test. 
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It is entirely possible that, for example, a third grade teacher with a 
heterogeneous group of students (heterogeneous with respect to the SAT) 
will have to be responsible for instruction on any one of a large list of 
NEC objectives. Recalling that the teacher is responsible for planning and 
the gathering of resources in addition to the delivery and assessment of 
instruction, this is an awesome task. Given that in light of the NEC em- 
phasis upon individualization it would be unacceptable to thwart the varia- 
bility of treatment to students, and given the problem with planning and 
delivery as stated above, it is essential to help clusters (teachers) anti- 
cipate the instructional needs of the incoming students and the range of 
these needs. More than anticipate, it will be necessary to provide planning 
support such that the variability any one teacher must accommodate is mini- 
mized (or held to an accomplishable level) while maximizing the range of 
student needs met. It is trusted that this "statement of probleirf' lays the 
groundwork for the development of local criterion profiles. 

The local criterion profile would be based upon some operational facet 
of the school. Initially, in the case of the NEC project, there will be a 
criterion profile for each grade level. In the case of the first grade, 
the criterion profile will be a reflection of those skills deemed necessary 
for the student to benefit from second grade exposure (or before he moves 
on to the second grade profile). The preceding parenthetical segment im- 
plies that the student who is physically located in a first grade cluster 
and who has finished his first grade criterion profile may either be moved 
to a second grade cluster or, if the teachers can handle the situation with- 
in the first grade, be moved on to the second grade criterion profile. 

Having criterion profiles will allow the clusters to anticipate the minimal 
instructional load respective to a group of incoming students. With help, 
this would allow the teachers to divide the instructional load such that 
each teacher will have a manageable load. Host likely this analysis and 
assignment will be performed several times throughout the instructional 
year (most precisely, as often as the array of student needs in conjunction 
with the ability to deliver instruction dictates). 

The obvious question is how to decide, at least initially, which skills 
(objectives) will be included in which local criterion profile. Two kinds 
of inputs will be used: (1) the previous year's performance data, and 
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( 2 ) the judgments of teachers and CAPs. The evaluation staff is conducting 
analyses of previous performance data^ in two ways: (1) mastery of which 

NEC objectives is critical to success on the external criterion (for the 
moment call these "critical objectives"), and (2) mastery of which antece- 
dent NEC objectives was related to mastery of the "critical objectives." 

For example, it has been ascertained that the student's level of mastery in 
"NEC language usage" is highly related to his level of performance on the 
external criterion (generally in grades A-6) • It would be an oversimplifi- 
cation to set mastery of NEC language usage as an immediate goal for Ath, 
5th, and 6th graders in general, for it is known that students who accom- 
plish a number of language usage objectives have also accomplished a number 
of antecedents (correlates), for example, mastery in structural analysis 
and phonetic analysis, which was in turn preceded by mastery in oral lang- 
uage, written language, and perception. As indicated above, it would be 
2 

unwi se to use only the previous year's SAT performance data in establish- 
ing local criterion profiles. For one reason, one would be limiting the 
set of objectives to those things measured (perhaps those things measured 
adequately) by an achievement test. For another, it would be failing to 
take into account the judgment of NEC instructional personnel as to what 
should be taught. 

Table 1 has been included in this report as an example of one type of 
data analysis that is going into the construction of the local criterion 
profiles. The data in Table 1 indicate that the 1971 first graders worked 
in seven NEC areas. Recalling that the expected level of performance for 
the end of first grade was 18 months of achievement, one can search through 
Table 1 for those levels of NEC mastery which were associated with grade 
level performance on the external criterion. 



^These analyses will be presented in detail in a project memo now in prep- 
aration for fall ('70 submission. 

2 

As a further contraindication, analyses have revealed that while success 
on some of the NEC objectives does not directly relate to performance on 
the external criterion, success on these objectives does relate to success 
on other NEC objectives which in turn relate to success on the external 
criterion. 
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TABLE 1 



Grade 1, 1971 » Average Performance (Grade Level Equivalents, Expected =* 18) 
on each Sub test of the Stanford Achievement Test PI Form X as a Function of 
Performance on NEC Objectives (Farthest Objective Mastered in the Designated 
Area) 

Farthest 







Objective 


Percent of 


Word 


Para. 






Word 






Area 


Mastered 


Students 


Read 


Mean. 


Voc. 


Spel 1 . 


Stdy . 


Arith. 


Oral 


Lang. 


0-3 


21.2% 


14 


14 


16 


14 


17 


15 


1 1 


11 


4-6 


30.0% 


16 


16 


18 


18 


19 


16 


1 1 


11 


10-13 


7.5% 


17 


17 


20 


16 


19 


17 


1 1 


II 


16 


41.3% 


19 


19 


24 


19 


23 


20 


Wri tten Lang. 


0-1 


42.5% 


15 


15 


16 


14 


18 


15 


1 1 


II 


2-3 


15.6% 


17 


17 


22 


17 


20 


19 


1 1 


II 


4-8 


41 .8% 


19 


19 


23 


21 


23 


19 


Percept. 


0-12 


20.6% 


14 


14 


15 


13 


17 


15 


1 1 




13-14 


20.0% 


15 


15 


17 


14 


16 


15 


1 1 




18-20 


59.4% 


19 


19 


22 


20 


23 


19 


Phon 


. Anal. 


0-2 


56.9% 


16 


16 


17 


15 


19 


16 


1 1 


11 


4/10/12/ 


43.1% 


19 


19 


23 


20 


22 


20 


Pre. 


Numb. 


0-11 


13.1% 


17 


17 


16 


13 


19 


16 


1 1 


II 


12-19 


22.5% 


16 


15 


18 


16 


19 


16 


1 1 


II 


20-22 


64.4% 


18 


18 


21 


19 


21 


18 


Read 


Write Count 


0-6 


26.3% 


15 


15 


15 


12 


16 


14 


1 1 


H n 


7-14 


26.3% 


16 


16 


17 


17 


19 


16 


1 1 


ii n 


15-22 


33.8% 


19 


19 


20 


21 


23 


18 


1 1 


ii ii 


28-31 


13.8% 


19 


18 


32 


19 


23 


24 


Add 


Sub. 


0-4 


63.1% 


16 


16 


17 


16 


18 


15 


ii 


ii 


8-9 


20.6% 


17 


16 


18 


18 


21 


17 


1 1 


n 


15-16 


16.3% 


22 


23 


34 


21 


17 


26 



ii n 
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Organization for Instruction 



The topic of concern here is the organization for instruction, speci- 
fically as it relates to planning for instruction. Major emphasis has been 
placed upon the implementation of the cluster concept as the basic Instruc- 
tional unit in the NEC project. Planning for and the delivery of Instruction 
is to occur within the cluster. The tone of the evaluation here is that 
during the fourth year of the project much more emphasis must be placed 
upon behavioral ly defining the role of the cluster and evaluating the degree 
of implementation of that role and, subsequently, evaluating the impact of 
that role upon student performance on the NEC objectives. 

Globally stated, the cluster Is intended to provide flexibility; the 
flexibility in turn is to support or promote individualization of instruc- 
tion. Ignoring mode of Instruction, the concept of "flexibility" can be 
brought to bear upon three activities: (1) deciding which objectives are 

to be taught at which time, (2) deciding to whom they are to be taught, 
and (3) deciding by whom they are to be taught. These three activities 
focus upon the major instructional resource, the teachers' effort. As dis- 
cussed on page 20 the NEC teachers have reported that the problem of moni- 
toring and delivering instruction to a group of students with diverse pro- 
files is becoming quite difficult. The central theme introduced here is 
that the problem can be somewhat ameliorated by more effective cluster plan- 
ning (which all relates back to behavioral ly defining the role of the cluster). 

It is essential to be able to demonstrate that the cluster (consisting 
of extra teacher service at substantial expense) clearly facilitates the 
teachers' ability to manage and implement individualization of instruction 
which subsequently leads to student performance on the NEC objectives. 

At this time it is not "objectively clear" that systematic cluster 
planning Is taking place in the NEC project. The evaluation staff recommends 
that each cluster (in conjunction with the CAPs and evaluation staff) derive 
a formal statement outlining the operational rationale for each of the clus- 
ters. It Is not suggested that there is to be only one type of operating 
rationale across the project. The only commonality among cluster operating 
rationales Is that they clearly address how the following decisions are 
made. 
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1. Which objectives are to be taught at which time. 

2. To whom they are to be taught. 

3. By whom they are to be taught. 

Once the operating rationales are developed for each cluster, observations 
will be made to establish that the defined rationale is implemented. Along 
with the observations of implementation, the teachers will be Interviewed 
as to the strengths and weaknesses of their particular method of operation. 

The final evaluative step will be to ascertain the relationships be- 
tween the way in which the different clusters operate and the extent of 
student performance on the NEC objectives. 

Relationships Among Teachers and Supervisors 

The acceptance and visibility of the NEC instructional objectives have 
been instrumental in fostering a cooperative and supportive relationship 
among the NEC teachers and administrative staff. As compared to situations 
where supervisory/instructional interactions are formalized visitations 
loaded with anxieties and ambiguities, both the teacher and the administra- 
tor In the NEC project are aware of the instructional needs of the concerned 
students. Likewise, both are aware of the progress made on the instruction- 
al objectives. The Individual and cluster performance profiles are the 
mechanisms which bring about this mutual diagnosis of progress. The key 
factor here is that the objective diagnosis of progress allows either the 
teacher or the administrator to initiate contact; and further, It is clear 
to both whether or not the specific problem has been addressed and resolved. 
The teacher is aware of whether or not she has been given substantive help 
and the administrator is aware of whether or not she has been effective at 
isolating and resolving an instructional problem. 

AccountabiJ^^^^oJl^iiidSIl^EJISflLSSS. 

In light of the current interest in performance contracting, an impor- 
tant and relevant outcome of the NEC project is the verbalized willingness 
of teachers and administrators to be accountable for student success on the 
NEC Instructional objectives (the reader should take care to contrast the 
preceding with accountability for a standardized test score). Several cri- 
tical factors underlie the acceptance of accountability by teachers and 
administrators in the NEC project. 2<J 
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1. The Instructional objectives were developed by and with the ap- 
proval of representatives of teachers and administrators. The initial set 
of objectives was the result of Inputs from all teachers and administrators. 

2. The performance measures (ORTs and CETs) were developed and ap- 
proved by representatives of teachers and administrators. As with the 
instructional objectives, the initial pool of items for the CETs and ORTs 
was the result of inputs from all teachers and administrators. 

3. An information support system has been developed (individual and 
cluster profiles) to facilitate instructional planning, delivery, and eval- 
uation. In cooperation with teachers and administrators this feedback sys- 
tem is undergoing continual modification in order to provide additional or 
more sophisticated support to the instructional staff. 

k. The entire process (l.e., objectives, tests, profiles, roles, etc.) 
is geared to promote and respond to needed changes. Mechanisms are immedi- 
ately available to teachers and administrators through which recommended 
changes or criticisms are responsively processed. 

f>. Data analysis procedures have been implemented which ascertain two 
types of relationships. First, analyses of the relationships among student 
performance on the NEC objectives give the teacher a feel for the implica- 
tions of current performance in relation to projected instructional activities. 
The relationships among stud«?!".t performance on the NEC objectives Is the 
"heart" of the feedback system, for it is instruction on these objectives 
which the teacher has under her control. (Contrast the degree of visibility 
and control the teacher has over student success on the NEC objectives with 
the at least obscure or indirect control of a student's score on a nationally 
normed standardized achievement test.) Second, through analysis of the re- 
lationships among student performance on the NEC objectives and performance 
on the external criterion (Stanford Achievement Tests), teachers, adminis- 
trators and particularly "central level" administration are provided assur- 
ance that "their local objectives" have external credibility. The authors 
feel obligated to assert that the second type of relationship is at best an 
insufficient degree of external validity. Meaningful credibility for an 
isolated objective should be based on the fact that it enables a student to 
benefit from further related instruction or, In general, that instruction 
on the objective enables the student to cope with his environment. 
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The authors strongly urge the reader to consider the NEC process as an 
alternative to the current "character" of performance contracting. We are 
not suggesting that the NEC objectives or mastery tests be adopted per se, 
but that the process by which they were adopted is both a general izabie and 
highly acceptable way in which to arrive at instructional accountability. 
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